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Recruiting schools and kids to participate in drug prevention evaluations 
is no longer, if it ever was, a “done deal.” Since the mid-1980s, schools have 
been inundated with requirements for drug programs and requests to 
cooperate with program evaluations. As a consequence, both treatment and 
control schools are harder to find. In addition, many school administrators 
feel that their students have been “overstudied” and parents often look 
askance at questionnaires that seek information on sensitive topics such as 
drug use. Even if evaluators successfully pass the “recruitment barrier,” 
additional problems, such as the loss of schools or study participants over 
time, may threaten the study’s integrity. This paper examines the challenges 
facing evaluators of community-based programs when the design calls for 
assessing prevention efforts targeted at children or adolescents in school. 
It discusses the barriers to recruiting schools for the evaluation and keeping 
them in the study over time, and presents alternative strategies for over- 
coming those barriers. It then turns to the problems of getting and keeping 
student participants, focusing, in particular, on strategies for obtaining 
parental consent and minimizing attrition over time. 



Following the passage of the Drug-Free Schools and Communities Act in 1986, 
the federal contribution to state and local drug prevention sharply increased - from $189 
million in 1987 to $463 million in 1990. This monetary incentive fueled local adoption 
of drug prevention programs, as did the requirement that schools show they have 
developed a comprehensive drug education plan before they can receive federal educa- 
tion funds. Many states have passed laws requiring schools to implement drug educa- 
tion, thereby enhancing the demand for prevention programs. 

Although schools across the country have acquired drug prevention curricula, few 
have been evaluated (U.S. GAO, 1993). Federal agencies have proposed broader 
evaluation requirements, but federal funding for careful evaluations has been limited. 
At the same time, the logistics of carrying out credible evaluations that meet accepted 
scientific standards have become more complex. These standards call for the ability to 
generalize to diverse school and community environments, to assess program effects over 
several years, to recruit enough schools and or communities to be able to detect pro- 
gram effects on rare behavior, and to carry out random assignment of schools to treat- 
ment and control conditions (Ellickson & Bell, 1992; Murray & Hannon, 1990). 

Challenges to Recruiting and Retaining Schools for Prevention Evaluations 

Thus evaluators need to convince gatekeepers from widely different school en- 
vironments to cooperate with study requirements and to do so over several years. Today, 
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we are much less likely to have the luxury of working solely with schools that know 
and like us. As the need for recruiting multiple and diverse schools increases, we are 
also more likely to come up against conflicts between the needs of the research and the 
organizational priorities and interests of the schools. 

Schools and school districts are bureaucracies. Like bureaucracies everywhere, they 
must satisfy their own organizational imperative for institutional survival and 
maintenance over time. Intrusions from the outside are more likely to be accepted if 
they enhance the school’s organizational needs and much less likely to be accepted if 
they pose a threat to the institution. 

Hence, recruiting schools for evaluation research is “essentially political in nature” 
(Hansen, 1991). Knowing the terrain — understanding what schools perceive as poten- 
tially costly and what they view as beneficial — is central to successful recruitment efforts. 
It is also essential to successful retention; conflicts between the research and school needs 
and priorities do not miraculously disappear once schools have agreed to participate 
in a study. Among the more common and challenging problems that evaluators may 
need to anticipate and overcome are: 

• conflicts with educational priorities and routines; 

• resentment of burdensome demands on school personnel; 

• concerns about respondent burden; and 

• concerns about negative publicity or parental complaints. 

Conflicts with Educational Priorities and Practice 

In an era when schools are increasingly berated for inadequate student achieve- 
ment, testing a new curriculum that may take time from the “three R’s” may appear 
unwarranted. That was the reason that one urban school district declined to participate 
in Project ALERT, a multisite drug prevention evaluation involving 30 schools from 
eight California and Oregon school districts (Ellickson, Bell, Thomas, Robyn, & Zellman, 
1988). Under court order to equalize student achievement across different racial and 
ethnic groups, the district felt that it could not justify any additional incursion on tradi- 
tional academic activities. 

Once a school has already invested in a drug prevention program, it may be reluctant 
to replace that program with something else. As more schools have adopted programs, 
fewer still need them. Thus researchers who want to test new or improved models have 
found it more difficult to locate districts interested in trying a different “product” or 
“brand.” They have also found it more difficult to locate control schools that do not 
already have a program or are willing to drop what they do have for the duration of 
the test. 

In addition, the program itself— or its implementation schedule — may conflict 
with established school routines and priorities. If the school’s teachers are to teach 
the lessons, they need to “buy into” the program and receive appropriate training. 
Teachers who resent delivering a program with which they are unfamiliar or that they 
consider superfluous may consciously or subconsciously undermine it during implemen- 
tation. If outside health educators teach the lessons, their disciplinary and teaching style 
may conflict with and disrupt the regular teacher’s mode of operation. Moreover, 
programs with many lessons (eight or more) may be difficult to schedule within a single 
semester if they must be spaced at weekly intervals. Schools that typically teach health 
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education modules within a 2- or 3-week period find weekly schedules particularly 
challenging. 

Even when the research program involves evaluating an existing program, educa- 
tional and research goals may come into conflict. For example, some teachers resent 
the time that data collection takes from regular classroom activities, feeling that the 
loss of even one or two classroom periods has a negative impact on educational goals. 

Extra Burdens on School Personnel and Respondents 

Prevention program evaluations inevitably place extra burdens on school personnel 
and student respondents. Teachers may be asked to help with the mechanics of data 
collection, to hand out and collect consent forms, to help identify students who lack 
permission to participate, and to explain the program to concerned parents. They may 
also need to “learn” a new curriculum and attend special training workshops to do so. 
Incursions on the time of school administrators and office personnel often include sending 
letters to parents that inform them about the research (and seek consent for their child’s 
participation), arranging schedules for data collection, arranging parent meetings, and 
escorting nonparticipating students to other school venues. Although no single demand 
is likely to cause rejection, fear of multiple burdens may well do so. 

In some districts or schools, the number of research projects that involve student 
surveys or testing may be at the saturation point. For example, one district said they 
had placed a moratorium on new research projects because they had already approved 
over 200 independent studies (personal communication, Lee Beringer, Pacific Institute 
for Research and Evaluation). Schools with high proportions of minority students tend 
to be at risk for slipping into the “overstudied” category, as are schools close to univer- 
sity campuses. Whether over- or understudied, schools are particularly likely to balk 
at projects that take up more than one classroom period for student data collection. 

Concerns About Negative Publicity and Parental Complaints 

For drug prevention evaluations, concerns about asking sensitive questions and the 
potential for negative publicity are intertwined. All drug use, including smoking and 
drinking, is illegal for minors. Hence some schools fear negative parental reactions to 
questions about drug use and other sensitive behavior. Some also worry about publicity 
related to the school’s participation in drug use research, fearing that media reports may 
give them a “bad school” image. These concerns were particularly salient in the early 
1980s, leading many districts to require assurances of anonymity and a low public pro- 
file as a condition for participating in drug use research (Ellickson, Bell, et al., 1988). 1 

Strategies for Improving Recruitment Outcomes 

The key to successful recruitment rests on the evaluator’s ability to present a credi- 
ble case for the research and to establish the staffs legitimacy as professional and reliable 
experts for carrying it out. Having a prior relationship with key gatekeepers in the school 
district helps the evaluator get in the door. If the research group has worked in the district 
before and developed a reputation for cooperating with school officials, the odds of 



'Since the late 1980s, however, drug use surveys have become commonplace and parental concerns about 
them have receded. In contrast, parents have mounted campaigns against surveys of adolescent sexual behavior 
in many communities and, in at least one instance, have sought a court injunction to block survey administration 
(personal communication, David Kanouse, RAND). 
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obtaining approval for a new study are substantially increased. Similarly, knowing an 
influential school board member or administrator may increase the odds of getting a 
hearing, particularly in smaller districts. 

Nevertheless, many projects have successfully recruited multiple districts and schools 
for evaluation studies without having the advantage of prior experience or personal con- 
tacts in them. Instead, they develop and nurture the requisite relationships during the 
recruitment process (Ellickson, Bell, et al., 1988; Goodman, Smith, Dawson, & Steckler, 
1991). In the following sections, we give examples of both approaches to recruitment. 

Finding Recruitment Staff with Appropriate Skills 

Recruiting schools for prevention evaluations is time-consuming and costly. Each 
district may have different gatekeepers (superintendents, principals, teachers, school 
board members, representatives of community groups) with varying agendas. Recruit- 
ment staff have to be prepared to meet with different district representatives and to an- 
ticipate and address their concerns. This may take more than one face-to-face visit. In 
addition to brochures describing program benefits and requirements, staff may also need 
to develop lengthier mini-proposals. They will definitely need to put essential aspects 
of the participation agreement in writing (Murray et al., 1992). Once approval has been 
given, the job of developing and maintaining cooperative relationships with participating 
districts does not disappear. Following through on initial promises is essential. Becom- 
ing aware of emerging problems and working to resolve them helps smooth the evalua- 
tion process and avoid crises that might precipitate district or school withdrawal. 

Much rests, therefore, on the skills of the recruitment team and their willingness 
to do the necessary upfront planning and follow-through. Projects with multiple sites 
dispersed across several counties and states are likely to need a team of negotiators with 
members who concentrate on different locations. In the following example, we see how 
one project chose its negotiating team for a prevention trial (Project ALERT) that re- 
quired enrolling schools from a predesignated pool of West Coast districts. The descrip- 
tion focuses particularly on the qualities that make people effective at recruiting schools 
and working with them over time. 

Because the project’s recruitment director lacked existing contacts in the candidate 
districts, she hired staff to serve as local contacts and negotiators (Ellickson, Bell, et 
al., 1988). She wanted people who could build long-term relationships in the district, 
act as troubleshooters for identifying and resolving problems before they became crises, 
and represent the interests of both the research and the district. She looked first for 
individuals with school expertise and good negotiating skills — people who had worked 
in or with schools in the past, had a good understanding of the organizational concerns 
and perspectives of school administrators and teachers, and were sensitive to their needs. 
She also looked for people who were flexible (were willing to work with schools and 
accommodate reasonable demands that would not compromise the research), believed 
in the importance of evaluating prevention programs, and had demonstrated a capacity 
to follow through on their commitments. 

To get people who fit — or nearly fit — this daunting job description meant relying 
on personal networks, considering people that project staff had already worked with 
or had heard about from a well-regarded source. Thus the project sacrificed propinquity 
for having the “right profile.” Although only one of the site coordinators lived in a can- 
didate district, they all had to be available for face-to-face presentations in potential 
sites within their geographic area. They also had to be prepared to conduct continuing 
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negotiations on their own (i.e., without the assistance of the recruitment director). Once 
the final sites had been selected, all of them spent several weeks on site working out 
scheduling, teacher training, and the myriad details associated with program implemen- 
tation and data collection. During data collection and program delivery, which took 
place over a period of 4 months, all of them were physically on site at least 2 days a week. 

An alternative strategy used in a smoking prevention dissemination study involved 
the following steps: (1) locating local intermediaries to make the initial contact with 
district superintendents, (2) sending members of the research team to meet with the 
superintendent and other school officials (and schedule additional meetings, if necessary), 
and (3) identifying a district-appointed liaison with responsibility for letting the re- 
searchers know what additional information the district desired (Goodman et al., 1991). 
Criteria for selecting the local intermediary included knowledge of the research team 
or project, agreement with the project's goals, and credibility with and access to the 
superintendent. However, although the number of meetings held between project and 
district representatives was statistically associated with the decision to join, having an 
intermediary was not (Goodman et al., 1991). Hence the willingness of the research team 
to commit substantial time and effort to recruitment (plus their research expertise) ap- 
peared to be a crucial factor in signing up schools for the study. 

Making a Credible Case and Establishing Legitimacy 

The first step in recruitment planning involves designating a pool of districts and 
schools that meet the project's need to test the program in particular or diverse school 
and community environments (e.g., urban, suburban, or rural communities; large or 
small schools; those with high or low minority populations, etc.). 2 After gathering in- 
formation on who the key gatekeepers in candidate districts are, initial contacts may 
be made by phone or letter. As noted above, having a local intermediary make that 
first contact facilitates obtaining access to school officials and bolsters the study's 
legitimacy (Goodman et al., 1991 ; personal communication, Royer Cook, ISA). Getting 
supporting letters from educational or other professionals respected by district officials 
does the same (Murray et al., 1992). 

However, neither supporting letters nor supportive local contacts are sufficient for 
establishing the credibility of the project and its research team. Fulfilling that require- 
ment usually takes at least one visit to candidate sites that have expressed some interest 
in participating (Goodman et al., 1991). It also requires making a strong, but balanced, 
case for the research, both verbally and in writing. Hence before any contacts are made 
in candidate districts, it is essential to clarify what you will need from the schools and 
what you, in turn will provide to them. This process involves three components: (1) 
laying out precisely what the research will require of each district and school; (2) show- 
ing what the schools will gain from participating; and (3) seeking additional ways to 
maximize the benefits and minimize the costs of involvement. 

Clarifying the Research Requirements 

At the outset, projects need to decide which aspects of the research design are non- 
negotiable and which can be altered. In Project ALERT, for example, we required that 



2 For different methods of identifying the relevant district pool, see Murray et al., 1992; and O’Hara et 
al., 1991). 
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districts allow us to assign schools randomly to treatment or control conditions and con- 
duct data collection (written questionnaires and physiological tests) with participating 
students over a 6-year period. Districts also had to forgo their own drug program in 
the treatment schools (if they had one) and avoid influencing the climate regarding drug 
use through atypical drug prevention efforts. In addition, treatment schools had to ac- 
commodate the program’s weekly implementation schedule (eight lessons delivered in 
mixed gender classrooms during grade 7; three lessons during grade 8) and all schools 
had to agree to using passive consent procedures. 

Each of these requirements carried some risk for the schools, or disrupted normal 
school procedures, or placed additional burdens on school personnel. For example, 
schools ran the risk of not receiving the new program (being a control), while still being 
involved in data collection. Those assigned to the treatment condition would have to 
drop any existing programs for the cohort involved in the experimental test; if they lacked 
an existing program, they would need to find time for 1 1 classroom lessons over a 2-year 
period. All schools were also subject to the burden of using school time for data collec- 
tion and potential parental concerns about data privacy or negative publicity, while many 
treatment schools had to alter their normal schedules for delivering special health units. 

The most burdensome of these costs turned out to be those that meant altering 
existing school priorities or practice. Of the 11 districts that rejected Project ALERT, 
5 already had a drug prevention program they did not want to drop and 4 either could 
not accommodate weekly scheduling within one semester or felt the program would take 
away time needed for academic activities. Only one district objected to random assign- 
ment and another objected to collecting physiological samples. 3 Nevertheless, school 
officials need to be informed about all of the potential costs at the outset. Otherwise, 
they cannot make an informed calculation of whether the project’s benefits outweigh 
its costs. Moreover, surprises that arise after agreements have been reached undermine 
credibility and may lead to subsequent withdrawal from the study. 

It is also essential that the recruitment team and anyone else involved in negotiating 
for the project be clear about study requirements and potential costs before initial con- 
tacts with school people are made. Othewise, schools may request deviations from the 
research protocol that you (or your representative) cannot easily classify as acceptable 
or not acceptable. Is acceptance of random assignment a nonnegotiable requirement 
for school participation or can the project live with something less than a classic ex- 
perimental design? If physiological samples are to be collected, does every school have 
to go along with it? Is the project committed to passive or active consent procedures? 
If the former, can it afford to implement active consent in some schools? Can the lessons 
be bunched together or do they need to occur weekly? These, and other issues, are likely 
to arise during the recruitment process. Although not every request can be anticipated, 
recruitment staff need a clear understanding of when they can make concessions, when 
they cannot, and when they need to get further clarification from the project leader. 

Maximizing Benefits to the Schools 

Project staff also need to provide schools with reasons to participate in drug preven- 
tion evaluations that offset the costs of involvement. A major incentive, of course, is 



Experiences with numerous projects in the midwest also indicate that few districts or schools decline 
to participate because of the randomization requirements (personal communication, David Murray, Univer- 
sity of Minnesota). 
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getting a free program that may benefit substantial numbers of students. Many evaluators 
have also hired and trained outside health educators to teach the program, thereby elimi- 
nating the costs to schools of training regular teachers. They then offer both the program 
and subsequent training to regular classroom teachers in all district schools after the ex- 
perimental test. We have found that the randomization requirement is rarely an obstacle 
to participation when control schools know they will receive the program within a year. 

A more serious obstacle, however, is the requirement for treatment and/or control 
schools to drop existing prevention programs (to which they have already committed 
money and time) during the experimental or demonstration period. One solution is to 
allow control schools to retain existing programs as long as they do not duplicate the 
one being tested; this dilutes a program/no program trial but has the advantage of pro- 
viding a more realistic (and conservative) test of the new model. For treatment schools, 
emphasizing the limited nature of the “hiatus” period (that dropping an existing program 
applies only to the experimental cohort and that it can be resumed after the test period) 
often diminishes concerns about wasting an existing investment. 4 Another offsetting 
factor is the opportunity for staff to get additional training and health education 
experience. 

An additional benefit is that offered by the research itself —the opportunity to partici- 
pate in an effort that will further understanding of what works and for whom and the 
chance to get current, albeit aggregate, information on student drug use and attitudes. 
Most districts find such information useful for assessing district needs and welcome the 
opportunity to get the data at no cost. However, evaluators need to be clear that in- 
dividual data will not be provided and that data aggregated to the school (versus the 
district) level must be in a form that does not allow for identification of individual schools. 

Finally, some projects offer other incentives for participation — equipment or funds 
for special school programs (drama, computer, education, science training, athletics) 
or monetary incentives linked to the size of the target population in the district and 
the proportion of surveys completed (O’Hara et al., 1991). Others offer “prizes” to 
classrooms with high participation rates (funds for a class trip, a classroom computer, 
etc.). These incentives take on added importance when the programs to be evaluated 
are already in place and the evaluator must convince treatment and comparison schools 
to participate in the reseach without the inducements of a free program or staff train- 
ing. When offering such incentives however, evaluators should take care to avoid under- 
mining the consent process. In-kind or monetary contributions at the whole-school or 
district level raise fewer concerns about undue pressure on individual students than in- 
centives linked to classroom participation. 

Minimizing Monetary and Nonmonetary Costs 

As noted above, costs to districts can be substantially reduced by hiring and training 
staff to deliver the prevention program. 5 However, if outside staff are more motivated 



4 This was our experience with Project ALERT. When we made an effort to convert rejections based on 
the requirement to drop existing programs, we improved recruitment outcomes. At the beginning of the 
negotiating process, however, we did not try to convert initial rejections. (Having a large pool of eligible 
districts, we felt we could afford to weed out “uncertain” districts before scheduling costly visits to them.) 

5 In one study that required regular teachers for program delivery, the six districts that declined all cited 
the costs in teaching staff and instructional time (removing teachers from the classroom during training, the 
cost of per diem substitutes, and the cost of the training workshops) as primary reasons for not participating 
(Goodman et al., 1991). 
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to deliver the lessons faithfully and better trained than regular teachers, doing so may 
reduce the “real world” nature of the test. Hiring outside staff also requires scheduling 
at least one meeting between the guest teacher and the regular classroom teacher to 
establish mutually acceptable guidelines for maintaining classroom order and upholding 
the classroom teacher’s authority and procedures. 

An alternative strategy involves convincing the teachers who normally deliver the 
school’s health education units that the new program is worth trying out and that they 
can deliver it effectively. Separate training workshops paid for by the project (or by 
other agencies interested in fostering program dissemination) are one way to accomplish 
this objective. 

Even if the direct costs of teacher time and training are covered through the project 
(or are not at issue for other reasons), program evaluations may still impose nonmonetary 
costs on schools and districts. These include extra time spent by school personnel on 
data collection or program implementation, class time lost from academic subjects, 
disruption of school schedules and routines, and goodwill spent “selling” or defending 
the program. 

Nonmonetary costs take on greater weight in the school’s decision calculus when 
the benefits of participating are indirect (involve knowledge that may help future genera- 
tions, but not the current student population) or otherwise limited (small monetary or 
in-kind contributions). To minimize these costs, projects should be prepared to: 

• take responsibility for all aspects of data collection, while limiting “in-school” 
data collection time required of students to one class period; 

• reimburse districts for exceptional incursions on the time of school personnel; 

• limit disruption of school schedules and routines; 

• hire staff whose function is to maintain contact with the district and resolve 
problems before they become crises; and 

• keep district and school staff fully informed of the research program’s progress. 

Taking responsibility for data collection , reimbursing exceptional costs. Project 
staff should maintain responsibility for all aspects of the data collection process — 
getting consent, supervising and conducting survey administration, collecting other data 
(such as physiological samples), identifying students without consent and omitting them 
from data collection, protecting data privacy, etc. However, some tasks, such as estab- 
lishing survey dates, getting lists of enrolled students, or sending letters to parents from 
the principal, inevitably require help from school personnel. Whenever possible, there- 
fore, staff should minimize the burden of those requests — for example, by drafting 
letters for the principal’s approval, paying for the use of school stationary, and mailing 
letters. Surveys or other forms should not take more than one class period for students 
to fill out; if other procedures are included (such as collection of saliva samples), survey 
items should be cut accordingly . 6 If unanticipated requests are made of school per- 
sonnel (e.g., extraction of data from school records), they should be reimbursed for 
their time. 



6 When projects need to administer surveys that take longer than one class period, scheduling afterschool 
administration is an alternative strategy. 
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In addition, standard requirements for protecting data privacy should be followed, 
including restricting access to student surveys to selected research personnel; using ID 
numbers, not student names, on individual questionnaires; restricting access to the link 
between names and ID numbers; and obtaining a Certificate of Confidentiality from 
the Department of Health and Human Services. These procedures help assure both 
schools and parents that individual student responses will not be made public or used 
against them in any way. 

Limiting disruption of school routines. Efforts should also be made to mitigate the 
disruption caused when program requirements involve altering normal school schedules 
or practice. For example, one school wanted to deliver Project ALERT in large, same- 
sex physical education classes, whereas the program required a small, mixed-gender 
classroom setting. The school allowed the project to form mixed-gender groups from 
the physical education classes; the project, in turn, hired extra teachers so that students 
could receive the program during their regularly scheduled physical education period. 

Hiring site coordinators to act as district liaisons. Site coordinators are especially 
important when the project has program implementation responsibilities: they can oversee 
teacher recruitment and/or training, develop schedules for program delivery, help 
reschedule classes missed because of school trips or bad weather, and respond to parent, 
school, or district concerns about the prevention program or the evaluation components. 
However, even when the project’s responsibility is confined to program evaluation, liaison 
staff play an essential role in paving the way for an efficient data collection process and 
fielding queries, complaints, or requests. Prior to Project ALERT’s baseline data col- 
lection, for example, a handful of parents in each district called with further questions 
about the evaluation. The ability of the site coordinator to describe the project’s exten- 
sive procedures for protecting data privacy both assuaged their concerns and relieved 
school personnel of the job. 

Keeping district and school staff informed. Keeping as many people as possible in- 
formed about the research helps to uncover possible problems and nip potential 
misunderstandings in the bud. Strategies include meeting with key stakeholders in the 
school and parent community, holding open parent meetings to explain the program, 
and responding quickly to calls made to the school or research office. 

In sum, successful school recruitment typically involves a top-down strategy — 
identifying the key decision makers in the district and convincing them of the evalua- 
tion’s merit . 7 Making a credible case ultimately rests on district perceptions that the 
evaluation’s benefits exceed its costs. Hence the recruitment staff has a complex task. 
They need to convey enthusiasm for the project and its likely benefits, while also present- 
ing a clear picture of project requirements and costs. They also need to understand school 
priorities and concerns and to make a sincere effort to accommodate requests that do 
not threaten the evaluation’s integrity. This ability to work with districts without sacrific- 
ing essential elements of the research design sets the stage for an ongoing collaboration. 
It signals that project staff have the expertise to carry out the evaluation efficiently and 
that they can be trusted to do it without inflicting surprise requirements on participating 
districts and schools. 



7 Once the recruitment phase is over, however, project personnel will be working with individual ad- 
ministrators, teachers, and clerical staff at each school. Hence the staffs ability to understand and anticipate 
concerns at the school level is crucial to maintaining an ongoing relationship. 
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Nevertheless, some districts will decline participation no matter how dedicated the 
staff or how favorable the benefit/cost ledger. This is most likely to occur when districts 
have priorities or problems that dominate their decision making. For example, one district 
contacted for a multisite study ostensibly balked at collecting physiological samples but 
in reality was beset by competing community group interests that made it difficult for 
district officials to make a decision and stick with it. After several meetings, staff decided 
to forgo continued efforts to reach an agreement. Efficient recruitment also rests on know- 
ing when to cut your losses. 

Recruiting and Retaining Student Participants over Time 

Getting and keeping schools in the evaluation is only part of the story; evaluators 
also face the challenge of getting and keeping student participants. One of the most 
intractable problems encountered in longitudinal research is that of nonresponse among 
members of the target population. School-based studies face three particularly trouble- 
some sources of nonresponse -lack of parental consent for student participation in the 
research, student absenteeism when data collection takes place, and students who transfer 
or drop out of participating schools. Each of these problems makes substantial inroads 
on sample size; each may also lead to disproportionately high losses of particular 
groups -minority children, those who do less well in school, and those who are “at risk” 
for engaging in problem behavior (Battjes & Bell, 1985; Ellickson, Bianca, & Schoeff, 
1988; Polk & Ruby, 1978). 

Maximizing Parental Response to Requests for Consent 

Requiring active written consent from parents (a signed form for every student par- 
ticipating in an evaluation) typically produces substantially reduced sample sizes at the 
outset -ranging between 40% and 60% of the target group (Donovan, Jessor, & Costa, 
1988; Josephson & Rosen, 1978; Lueptow, Muller, Hammes, & Master, 1977; Severson 
& Biglan, 1989). An active consent requirement also tends to result in underrepresenta- 
tion of groups that are important to include in drug prevention research — minorities, 
low achievers, and children who have already tried drugs (Kearney, Hopkins, Mauss, 
& Weisheit, 1983; Severson & Ary, 1983; Thompson, 1984). Low initial response rates 
and biased samples can jeopardize a study’s results from the beginning and call into 
question the wisdom of proceeding further. 

Although extensive follow-up efforts can substantially raise response rates, such 
efforts tend to be prohibitively expensive and time consuming for large research projects. 
One pilot study, carried out in 1984, found that it took 4 weeks and $25 per case to 
retrieve written forms from every parent (Ellickson & Hawes, 1989). The process involved 
multiple follow-ups and several different techniques for contacting parents — two mail- 
ings plus a “take-home” consent package, two parent meetings, and two waves of phone 
calls to nonrespondent parents with several contact attempts at each wave. Based on 
these results, the researchers estimated that achieving an 85% consent rate would cost 
about $90,000 for a sample of 6,000 students from 30 West Coast schools. Another 
active consent process, conducted in a single high school under heavy public opposition 
to the research, succeeded in retrieving written forms from approximately 65% of 2,500 
parents after extensive telephone follow-up, repeated mailings, and special outreach to 
students. The cost was substantially higher, amounting to about $45 per case (personal 
communication, Jennifer Hawes-Dawson, RAND). In contrast, conducting passive 
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consent proceedings with a sample of 6,000 Oregon students cost approximately $6,000 
or $1 per case (Severson & Biglan, 1989). 

Thus evaluators seek alternatives to active written consent. Two strategies that have 
proved successful include “passive consent” (where parents return a form only if they 
do not want their child to participate) and “active verbal consent” (where parents can 
give consent over the phone rather than having to mail in a signed form). 

Passive procedures yield participation rates ranging anywhere between 80% and over 
95% (Ellickson & Hawes, 1989; Severson & Biglan, 1989). When they involve multiple 
channels of communication (both mail and “take-home” consent packets) plus postcard 
reminders, the risk of failing to inform parents about the research is small. 8 In addition, 
studies suggest that the great majority of parents who fail to return a form under “passive” 
consent do so because of apathy, not because they oppose the research. Ellickson and 
Hawes (1989) found that only 4% of the parents contacted after not responding to passive 
consent mailings said they did not wish their children to participate. Severson and Biglan 
(1989) estimated that about 5% fewer parents are likely to give consent when the pro- 
cedures involve face-to-face meetings with the family as opposed to passive consent mail- 
ings. Moberg and Piper (1990) compared refusal rates from telephone contacts with 
those for mailed-in returns, finding that the former were about 5% higher. 

Nevertheless, many institutional review boards have rejected requests to use passive 
parental consent procedures for studies that collect sensitive data from children or 
adolescents. They typically cite concerns that some parents may not have received, read, 
or fully understood mailed material. 

An alternative process that involves accepting parental consent over the phone was 
recently tested in Wisconsin. After mailing materials that requested written consent, 
attempts were made to call approximately 1,000 nonrespondent parents, explain the 
research, and ask permission for the child to participate in a 4-year longitudinal study 
of nutrition and drug use behavior. Accepting telephone consent without written 
documentation from the parent increased the consent rate by 20%. Overall, consent 
was obtained from 87.5% of the parents of enrolled students and 96% were contacted 
(Moberg & Piper, 1990). Although these procedures are more costly than traditional 
passive consent methods, they offer additional assurance that parents have been con- 
tacted and had an opportunity to ask questions about the research. 

Reducing Nonresponse Due to Absenteeism and Student Transfer 

Nonresponse attributable to absences on the day of data collection can produce 
sample losses of 10% or more. As a group, absentees typically use drugs more frequently 
than students who are present during data collection (Brunswick & Messer, 1985; Kandell, 
Kessler, & Margulies, 1978; Pirie, Murray, & Luepker, 1988); hence retrieving as many 
absentees as possible is desirable. Make-up sessions, typically held within 1 to 2 weeks 
after the scheduled data collection at each school can substantially reduce this source 
of bias. They are designed to “recapture” students whose absence is caused by short- 
term factors — a brief illness, a trip to the doctor, or a sudden decision to skip school 
that particular day. 9 



following the above procedures with nearly 10,000 students, RAND received only one complaint that 
the parent had not known about the research after 4 waves of data collection (Ellickson & Hawes, 1989). 

^ake-up sessions are less likely to recapture students prone to chronic absenteeism or those suffering 
prolonged illnesses. 
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Table 1 shows the results of conducting make-up sessions for the schools that par- 
ticipated in Project ALERT. The initial make-up session, conducted when the students 
were in grade 7, reduced the absentee rate by more than two thirds (from 8.3% to 2.5%) 
and recaptured students who qualified as high-risk candidates for future drug use. Subse- 
quent efforts at data collection periods 2 through 6 yielded reductions of 50% or more. 
Overall, the make-up sessions retrieved close to 6% of the baseline sample at four of 
the five postbaseline waves. 



Table 1 

Results of Make-up Data Collection for Project ALERT 

(Percent of sample) 



Item 


Tl 


T2 


T3 


T4 


T5 


T6 


Absent before makeup 


8.3 


11.2 


7.5 


9.2 


9.8 


12.3 


Absent after makeup 


2.5 


5.5 


3.4 


3.3 


4.0 


5.3 


Percent reduction 


69.9 


50.9 


54.7 


64.1 


59.2 


56.9 



Note. Baseline sample is defined as enrolled 7th-grade students who filled out a survey at Tl. The T1 
calculations are based on the target population, all enrolled 7th-graders in participating schools. Because Tl 
and T2 makeups were conducted only in the 18 schools scheduled for the second (spring) wave of program 
implementation, the percentages reported for those two waves are calculated for the spring target population 
and the spring baseline sample, respectively. 



An even more significant threat to maintaining sample generalizability comes 
from subsequent student transfers out of the schools where the initial data collection 
was conducted. A meta-analysis of 85 substance abuse prevention studies showed that 
the mean proportion of students lost at postbaseline follow-ups ranged from 19% after 
3 months to 33% after 3 years (Hansen, Tobler, & Graham, 1990). Other studies have 
shown that transferees and school dropouts are disproportionately likely to come from 
groups that prevention programs seek to help -minority students, poor achievers, those 
from disrupted families with low socioeconomic status, and those with a higher pro- 
pensity to become drug users or abusers (Pirie et al., 1988). 

One method for improving retention rates involves tracking transferees and dropouts 
through the mail. In Project ALERT, 15% of the baseline sample— more than 1,000 
students— had transferred out of the participating schools within a year after the baseline 
seventh-grade data collection. The tracking procedures used in this study yielded surveys 
from two thirds of the transferees and cut the overall attrition rate in half (from 20% 
to 10%). As Table 2 shows, it also recaptured a substantial proportion of minority 
students and those with multiple at-risk characteristics — poor grades, prior deviance and 
drug use, and disrupted family life-styles. 

The overall tracking plan centered on a traditional mailed survey, with mail and 
telephone follow-ups plus monetary incentives (Ellickson, Bianca, & Schoeff, 1988). 
Within that overall strategy, staff incorporated an additional technique, obtaining in- 
formation about what new school the student was attending and, when successful, mailing 
the questionnaire to that school rather than to the last known home address. This strategy 
was designed to mitigate the problems of finding a highly mobile population of young 
adolescents, 90% of whom had moved at least once. Such mobility substantially reduced 
the likelihood of locating trackees at their old home addresses or finding a parent or 
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Table 2 

Tracking Results for Project ALERT 



Student characteristics 


Percent recaptured by tracking 3 


Disrupted family 5 


57 


Deviant c 


56 


Grades of C or lower 


49 


Black 


44 


Tried cigarettes 


49 


Tried marijuana 


44 


Tried alcohol 


43 



a Reflects the proportion of the loss before tracking that was recovered by the tracking process at the 
third data collection, one year after baseline. 

b Students who were not living with both natural (or “original’') parents. 

c Students who scored > .25 on a deviant behavior scale tapping truancy, vandalism, lying, cheating, and 
stealing. 



relative at those addresses. 10 Overall, the “new school” strategy enabled staff to find 
one third of the transferees (44% of the students located) and accounted for 43% of 
the completed surveys. Mail and telephone follow-ups accounted for the rest. Hence 
the “new school” strategy is an important component of the tracking process, but not 
an approach that can stand by itself. 

Having good background data on each student also helps to improve tracking results 
(Pirie et al., 1989). 11 Several types of data are useful in tracking younger adolescents. 
In addition to information on school transfers, these include having full names for both 
parents or guardians and getting the child’s and parents’ social security numbers, the 
child’s birthdate, and the names of third parties who might know of future family 
changes. Putting extensive efforts into collecting this background information pays 
substantial dividends; Pirie and colleagues (1989) describe three different studies that 
did so and subsequently located 80% or more of students in the tracking pool. 

Conclusions 

Researchers and funding agencies have increasingly recognized that credible evalua- 
tions of school-based prevention programs require long-term assessment of students from 
multiple schools and diverse community environments. However, requirements for large 
numbers, diversity, and multiple follow-ups make the job of recruiting and retaining 
both school and student participants more difficult. 

Schools and districts are much more likely to approve prevention evaluations 
when doing so fulfills their own organizational objectives (or at least does not conflict 
with them) and does not impose heavy burdens or costs on school personnel and 
students. When the perceived ratio of benefits to costs is low or negative, successful 



10 Third-party information (names, addresses, and telephone numbers of relatives or friends who might 
know the student’s new home address) was also available, but this information was far from complete. 

u Once students have been located, offering individual monetary incentives for returning the survey helps 
to improve completion rates. Because students in the tracking pool have typically consented to prior data 
collection efforts, such incentives are unlikely to distort the consent process. 
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recruitment is unlikely; conversely, a positive benefit/cost calculation raises the likelihood 
of approval. 

Hence a crucial prerequisite for successful recruitment involves making a credible 
case for the research -showing how the project will benefit the district and indicating 
willingness to avoid extra burdens on staff and students. At the same time, however, 
it is essential to maintain the integrity of the research design. Making a credible case 
also requires presenting a balanced view of the overall project — laying out precisely what 
districts will need to agree to (such as random assignment, asking students questions 
about sensitive subjects, using class time to collect baseline and multiple follow-up data, 
cooperating with consent procedures, etc.), and what those requirements may entail in 
terms of staff time and other potential costs. 

The delicate nature of this task requires finding people to negotiate with school 
districts who are sensitive to school concerns, understand the needs of the research, and 
will work at developing a long-term relationship with the schools. Whereas identifying 
local contacts who have credibility with key gatekeepers helps obtain access, projects 
also need liaison staff who will respond to school concerns and make sure commitments 
are filled throughout the life of the project. An alternative strategy involves hiring site 
staff who either live in the district or can commit to being there during crucial negotia- 
tion and implementation periods. 

Maximizing student participation and retention over time requires a different set 
of skills — expertise in designing and carrying out strategies for obtaining parental and 
student consent and in minimizing nonresponse attributable to absentees, school 
transfers, and school dropouts. Using passive consent procedures or active verbal con- 
sent helps reduce sample loss attributable to lack of parental consent, while holding 
makeup sessions and tracking transferees and dropouts by phone or mail helps retrieve 
surveys from students who were absent on the day of data collection or no longer at- 
tend the school where data collection took place. 

Nevertheless, the site liaison can and should contribute substantially to the process 
of getting and keeping students by responding to parent queries, helping to arrange 
makeup schedules, and working with schools to identify the nonrespondents who need 
special follow-up. Building and maintaining cooperative relationships with school and 
district personnel is central to successful implementation over time, as well as to suc- 
cessful recruitment. 
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